Discord message links - #135
Open
ArnabChatterjee20k wants to merge 21 commits into
Open
Conversation
Adds config-driven MCP support so the Claude agent can query the
production Appwrite Cloud (via the Appwrite MCP server) instead of a
local stack when investigating user-reported issues.
- New McpServerConfig under ProviderConfig.mcp, keyed by server name,
with a per-server sources list (default use: helpscout only; empty
means all sources).
- Runner renders matched servers to a private 0600 .mcp.json temp file,
passes --mcp-config/--strict-mcp-config, and auto-allowlists
mcp__<server> tools for both fix and Q&A runs. Secrets stay in env
via ${VAR} expansion; runs without an issue never attach MCP.
- Threads issue source through the execute path; wires mcp through both
runner build sites (lib.rs, main.rs).
- Example config + unit tests (round-trip, source gating, rendered
config perms/shape).
- Pass issue source as &str via as_str() in verify/reply paths. - Write rendered MCP config to the open temp handle instead of reopening the path (avoids Windows exclusive-lock failures). - Validate exactly one of command/url per server; skip and warn otherwise so strict MCP loading never sees an ambiguous transport. - Add per-server tools allowlist: scope to mcp__<server>__<tool> when set, else grant all via mcp__<server>. Lets read-only runs be scoped to read tools; read-only API key remains the enforced boundary. - Example config shows read-only tool scoping; config test covers tools.
- Read-only runs (Q&A/verify/reply) no longer get unscoped MCP tools; a server must declare an explicit `tools` allowlist to be usable there. Fix runs still default to all tools. Closes the read-only-boundary gap. - Validate transport consistency: reject `command` with a non-stdio type and `url` with stdio, not just presence, so --strict-mcp-config never sees a contradictory server. Added has_valid_transport() + tests. - Clarify render_mcp_config doc (temp filename, 0600 is Unix-only).
Read-only runs (Q&A/verify/reply) now draw tools only from a dedicated per-server readonly_tools list, never from `tools` (which may include mutating tools used by fix runs). Since a tool's capability cannot be verified at config time, the operator must explicitly list non-mutating tools for read-only use; with none listed, read-only runs get no MCP tools. Closes the remaining production-mutation path.
Asserts the url branch emits type/url/headers and omits stdio-only fields (command/args/env). Addresses review coverage gap.
- Reword the strict-mcp-config comment: flags are only added when a config is attached; no MCP flags when nothing matches. - Render tests read the temp file via reopen() instead of by path, matching render_mcp_config's Windows-safe handle write.
The API key is the access boundary, so a separate read-only tool list added complexity without a real guarantee. Collapse to one `tools` array (empty = all of the server's tools) applied uniformly to fix and Q&A runs. Drop readonly_tools.
The reproduce/verify stage already produces a structured verdict (root cause, impact, suggested fix, evidence) but it was only posted as a note and thrown away before the fix ran. Carry it on ProcessingInput and prepend it to the fix prompt context so the fix agent starts from a confirmed root cause instead of re-deriving one.
Three opt-in triage guardrails from the Cloudflare/Astro triage model: - verify_fail_open (ReplyConfig, default true): when the reproduce/ verify stage can't run (timeout/error/unsupported), setting this false asks the reporter for repro steps instead of forcing a fix. - fail_on_regression (EvaluationConfig, already existed): now wired. A successful attempt whose after-fix eval shows new failures or regressions is failed and retried instead of shipping the PR. - request_reporter_verification (ReplyConfig, default false): after a PR is created, ask the original reporter to confirm the fix resolves the issue on their end.
Drops request_reporter_verification and the post-PR reporter ping. Keeps the diagnosis-forwarding, verify_fail_open, and regression-gate guardrails.
The e2e config builder constructs ReplyConfig field-by-field, so the new verify_fail_open field must be set explicitly. Addresses greptile review comment on PR #132.
Adds opt-in evaluation.require_red_green (default false). When enabled and a test tool is detected: - Red phase: before the fix, a dedicated agent run authors a failing test only (no app code). The eval suite is re-run against the baseline; if no new test failure appears, the bug isn't reproduced and the attempt fails. - Fix phase: the fix prompt is told the failing test already exists and to make it pass without weakening it. - Green phase: the existing after-fix eval gate is forced on in red-green mode, so a test still failing after the fix fails the attempt. Adds EvaluationResult::has_new_test_failures() (test-category only) and a covering unit test.
The red-green phase previously ran invisibly inside Pending. Now it emits dedicated timeline events and issue decisions: - RedGreenStarted when the failing-test phase begins - RedConfirmed / red_green_not_reproduced for the red assertion - GreenConfirmed / not_green for the after-fix assertion Also records red_green action runs (red_confirmed / not_reproduced / green_confirmed / not_green) so the dashboard timeline reflects each step instead of showing only a stalled Pending attempt.
Greptile SummaryThe PR adds start/end Discord message links to retrieved discussion context and appends a compact reference list to notifier-delivered answers and replies.
Confidence Score: 4/5The PR is not yet safe to merge because a red-phase agent failure can still bypass the required reproduction gate and later be accepted as green. The red-phase error branch continues into the fix pipeline without recording or enforcing red confirmation, while later green evaluation only checks that no new test failures remain. Files Needing Attention: crates/claudear-engine/src/processing.rs Important Files Changed
Reviews (3): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
(Provide a description of what this PR does.)
Test Plan
(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work.)
Related PRs and Issues
(If this PR is related to any other PR or resolves any issue or related to any issue link all related PR and issues here.)
Have you read the Contributing Guidelines on issues?
(Write your answer here.)